Papers with counter-fitted embedding injection
GiBERT: Enhancing BERT with Linguistic Information using a Lightweight Gated Injection Method (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent pre-trained language models such as BERT have led to noticeable improvements in semantic similarity detection. |
| Approach: | They propose to explicitly inject linguistic information in the form of word embeddings into any layer of a pre-trained BERT. |
| Outcome: | The proposed method improves on multiple semantic similarity datasets and shows that it is beneficial and currently missing from the original model. |